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SYSTEMS AND METHODS FOR IDENTIFYING 
FOREIGN OBJECTS AND DEBRIS (FOD) AND DEFECTS 
DURING FABRICATION OF A COMPOSITE STRUCTURE 

5 COPYRIGHT NOTICE 

[0001] A portion of the disclosure of this document contains material 
that is subject to copyright protection. The copyright owner has no objection to 
the facsimile reproduction by anyone of the patent disclosure, as it appears in the 
U. S. Patent and Trademark Office patent files or records, but othenA^ise the 
10 copyright owner reserves all copyright rights whatsoever. 

FIELD 

[0002] The present invention relates generally to the fabrication of 
composite structures, and more particularly to systems and methods for locating 
15 foreign objects and debris (FOD) and defects during fabrication of a composite 
structure. 



BACKGROUND 

[0003] Composite structures have been known in the art for many 
20 years. Although composite structures can be formed in many different manners, 
one advantageous technique for forming composite structures is a fiber 
placement or automated collation process. According to conventional automated 
collation techniques, one or more ribbons of composite material (also known as 
composite strands or tows) are laid down on a substrate. The substrate may be a 
25 tool or mandrel, but, more conventionally, is formed of one or more underlying 
layers of composite material that have been previously laid down and compacted. 

[0004] Conventional fiber placement processes utilize a heat source 
to assist in compaction of the plies of composite material at a localized nip point. 
In particular, the ribbon or tow of composite material and the underlying substrate 
30 are heated at the nip point to increase the tack of the resin of the plies while 
being subjected to compressive forces to ensure adhesion to the substrate. To 
complete the part, additional strips of composite material can be applied in a 
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side-by-side manner to each layer and can be subjected to localized heat and 
pressure during the consolidation process. 

[0005] Unfortunately, defects can occur during the placenment of the 
composite strips onto the underlying composite structure. Such defects can 
5 include tow gaps, overlaps, dropped tows, puckers, and twists. Additionally, 
foreign objects and debris (FOD), such as resin balls and fuzz balls, can 
accumulate on a surface of the composite structure. Resin balls are small pieces 
of neat resin that build up on the surfaces of the fiber placement head as the pre- 
impregnated tows pass through the guides and cutters. The resin balls become 

1 0 dislodged due to the motion and vibration of the fiber placement machine, and 
drop on to the surface of the ply. If not removed, subsequent courses will cover 
the resin ball and create a bump in the laminate whereat there may be no 
compaction of the tows. Fuzz balls are formed when fibers fray at the edges of 
the tows and then break off as the tows are passed through the cutter assembly. 

15 The broken fibers collect in small clumps that fall onto the laminate and, if not 
removed, are covered by the next course. 

[0006] Composite laminates fabricated by fiber placement 
processes are typically subjected to a 100% ply-by-ply visual inspection for both 
defects and FOD. Typically, these inspections are performed manually during 

20 which time the fiber placement machine is stopped and the process of laying 
materials halted until the inspection and subsequent repairs, if any, are 
completed. In the meantime, the fabrication process has been disadvantageously 
slowed by the manual inspection process and machine downtime associated 
therewith. 

25 [0007] Recently, inspection systems have been developed that are 

capable of identifying defects in the composite structure during the fabrication 
process without requiring machine stoppage for manual inspections. Although 
such inspection systems have worked well for their intended purposes, the 
inventors have recognized that it would be even more beneficial if FOD could 

30 also be identified in-process during the fabrication of the composite structure. 
This, in turn, would eliminate the need for manual FOD inspections and the 
machine downtime associated therewith. 
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SUMMARY 

[0008] In order to solve these and other needs in the art, the 
inventors hereof have succeeded in designing systems and methods for 
identifying foreign objects and debris (FOD) and defects during fabrication of a 
5 composite structure. In one embodiment, the system includes at least one light 
source positioned to emit light that illuminates a portion of the composite 
structure with bright field illumination and that also illuminates another portion of 
the composite structure with dark field illumination. The bright field illumination is 
reflected differently by defects in the composite structure than from portions of 
10 the composite structure that are defect free. The dark field illumination is reflected 
differently by FOD on the composition structure than from surfaces of the 
composite structure that are FOD free. The system also includes at least one 
camera for receiving images of the illuminated portions of the composite 
structure. 

15 [0009] Further areas of applicability of the present invention will 

become apparent from the detailed description provided hereinafter. It should be 
understood that the detailed description and specific examples, while indicating at 
least one exemplary embodiment of the invention, are intended for purposes of 
illustration only and are not intended to limit the scope of the invention. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0010] The present invention will become more fully understood 
from the detailed description and the accompanying drawings, wherein: 

[0011] Figure 1 is a schematic view of a system for identifying FOD 
25 and defects during fabrication of a composite structure according to one 
embodiment of the present invention; 

[0012] Figure 2 is a perspective view of a system for identifying 
FOD and defects during fabrication of a composite structure according to another 
embodiment of the present invention; 
30 [0013] Figure 3 is a perspective view of a light source according to 

the embodiment of the system shown in Figure 2; 
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[0014] Figure 4 is a perspective view of a system for identifying 
FOD and defects during fabrication of a composite structure according to another 
embodiment of tlie present invention; 

[0015] Figure 5 is a perspective view of a light source according to 
5 the embodiment of the system shown in Figure 4; 

[0016] Figure 6 is a video frame capturing both FOD and a gap 
between tows as captured by one embodiment of the present invention; 

[0017] Figure 7 is a view of a computer display and selected user 
controls according to one embodiment of the present invention; and 
10 [0018] Figure 8 is a view of a computer display and selected user 

controls according to another embodiment of the present invention. 

[0019] Corresponding reference characters indicate corresponding 
features throughout the several views of the drawings. 

15 DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS 

[0020] Embodiments of systems for identifying foreign objects and 
debris (FOD) and defects during fabrication of a composite structure are 
generally indicated by reference numeral 10 in Figures 1, 2 and 4. As shown in 
Figure 1, the system 10 is positioned proximate a composite structure 22, which 

20 is generally comprised of a plurality of adjacent tows or strips 24 of composite 
tape. The strips 24 typically include a plurality of fibers embedded in a resin or 
other material that becomes tacky or f lowable upon the application of heat. The 
strips 24 are arranged on a work surface, such as a table, mandrel, or other tool 
26, and compacted with a compaction roller 20 (Figure 2) to form the composite 

25 structure 22 according to an automated collation technique, such as that 
described in U.S. Patent Application No. 10/068,735, filed on February 6, 2002, 
entitled "Composite Material Collation Machine and Associated Method for High 
Rate Collation of Composite Materials." The contents of U.S. Patent Application 
No. 10/068,735 is incorporated herein by reference in its entirety as if fully set 

30 forth herein. 

[0021] With further reference to Figure 1 , the system 10 includes at 
least one camera 12 and at least one light source 14. The camera 12 is 
connected to a processor 66 for interpreting the images the camera 12 captures, 
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or to a storage device 64 for storing the images, or both, as discussed more fully 
below. 

[0022] The light source 14 is positioned to emit light that illuminates 
a first portion 17 of the composite structure 22 with bright field illumination (also 
5 known as incident light) while also illuminating a second portion 19 with dark field 
illumination (also known as indirect light). More specifically, the bright field 
illumination "spills" over onto the second portion 19 of the composite structure 22 
and illuminates the second portion 19 with dark field illumination. 

[0023] The bright field illumination is reflected differently by defects 

10 in the composite structure than from portions of the composite structure that are 
defect free. For example, the bright field illumination reflecting off non-defective 
portions of the composite structure 22, and light that fails to reflect off of defects 
in the composite structure 22, or vice versia, creates visible images that can be 
captured by the camera 12. Details regarding systems and methods for 

15 identifying defects in a composite structure during fabrication thereof are included 
in U.S. Patent Application No. 09/819,922, filed on March 28, 2001, entitled 
"System and Method for Identifying Defects in a Composite Structui'e" and in U.S. 
Patent Application No. 10/217,805, filed on August 13, 2002, entitled "System for 
Identifying Defects in a Composite Structure". The contents of U.S. Patent 

20 Application Nos. 09/819,922 and 10/217,805 are incorporated herein by 
reference as if fully set forth herein. 

[0024] In addition, the dark field illumination is reflected differently 
by FOD on the composition structure 22 than from surfaces of the composite 
structure that are FOD free. For example, the dark field illumination reflecting off 

25 FOD, such as resin balls and fuzz balls, on the composite structure 22, and the 
dark field illumination that fails to reflect off the FOD-free portions of the 
composite structure 22, or vice versa, creates visible images that can also be 
captured by the camera 12, as discussed in detail below. 

[0025] As shown in Figure 1, the camera 12 is positioned near the 

30 composite structure 22 so as to capture images of the illuminated portions 17 and 
19 of the composite structure 22, which are typically immediately downstream of 
the nip point at which a composite tow is joined with the underlying structure. 
Alternatively, and as shown in Figure 2, a reflective surface 16 may be positioned 
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near the composite structure (not shown in Figure 2), and angled such that the 
reflective surface 16 reflects an image of the illuminated portions 17 of the 
composite structure. In one embodiment, the angle of the reflective surface 16 to 
the composite structure is about sixty-five degrees, but the reflective surface 16 
5 can also be positioned at any appropriate angle in order to reflect images of the 
illuminated portions of the composite structure to the camera 12. The camera 12 
then may be positioned to point toward the reflective surface 16 in order to 
capture the close-range images of the illuminated portions of the composite 
structure from the reflective surface 16. More than one reflective surface 16 may 

10 also be utilized in further embodiments of the present invention in which the 
reflective surfaces 16 cooperate in order to direct images of the illuminated 
portions of the composite structure to the camera 12. 

[0026] For composite structures having curved/contoured surfaces, 
an image of the composite structure is advantageously captured from a position 

15 as close as possible to the nip point in order to obtain an accurate representation 
of the composite structure for processing. Thus, the configuration illustrated in 
Figure 2 is particularly advantageous for capturing images of curved/contoured 
surfaces of the composite structure because the reflective surface 16 reflects 
images of the composite structure for the camera 12 to capture from a position as 

20 close as possible to the composite structure. In addition, this configuration 
permits the camera 12 to be placed further from the composite structure than the 
reflective surface 16, such that the camera 12 does not obstruct the functionality 
of other parts of the fiber placement device, or vice versa. Further, the reflective 
surface 16 can also provide a "square on" view of the area being inspected, 

25 which, in turn, can materially improve the ability to dimension the two gaps for 
pass/fail decisions. 

[0027] A wide range of cameras can be used including 
commercially-available cameras capable of acquiring black and white images. In 
one embodiment, the camera 12 is a television or other type of video camera 
30 having an image sensor (not shown) and a lens 13 through which light passes 
when the camera 12 is in operation. Other types of cameras or image sensors 
can also be used, such as an infrared-sensitive camera, a visible light camera 
with infrared-pass filtration, a fiber optic camera, a coaxial camera. Charge 
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Coupled Device (CCD), or Complementary Metal Oxide Sensor (CMOS), The 
camera 12 can be positioned proximate the composite structure 22 on a stand 
(not shown) or mounted to a frame 28 or similar device. In embodiments of the 
present invention that do not include a reflective surface 16, the camera 12 may 
5 be positioned approximately six inches from the surface of the composite 
structure 22, and mounted to the frame 28 by way of a bracket 30 and associated 
connectors 32, as shown in Figure 1 . 

[0028] In an embodiment that includes a reflective surface 16, 
however, the reflective surface 16 may be positioned approximately three inches 

10 from the surface of the composite structure 22, and the camera 12, pointed 
toward the reflective surface 16, may be positioned further away from the 
composite structure, as described above. In further embodiments of present 
invention, the reflective surface 16 may be positioned at other distances from the 
surface of the composite structure 22, such as from one to six inches, to 

15 accurately reflect ah image of the surface of the composite structure toward the 
camera 12. Reflective surfaces can also be utilized to allow the camera to be 
placed in an advantageous position which might othenwise be blocked by portions 
of the compaction roller 20 (Figure 2) and/or other parts of the fiber placement 
system. 

20 [0029] The connectors 32 may be rivets, screws or the like that 

mount the camera 12 to the frame 28 in a stationary position. Alternatively, the 
connectors 32 may be a hinge-type connector that permits the camera 12, light 
source 14, and associated assembly to be rotated away from the composite 
structure 22. This embodiment is advantageous in situations where other parts of 

25 the material placement device, particularly the parts located behind the camera 
12 and associated assembly, must be accessed, such as for maintenance, 
cleaning, or the like. 

[0030] Figure 2 illustrates an alternative embodiment of the hinge- 
type connector 32 that mounts the camera 12, reflective surface 16, light source 

30 14, and associated assembly (i.e., camera assembly) to the frame 28 by way of a 
bracket 30. A suitable fastener, such as a thumbscrew or any other fastener that 
may be removed or loosened with relative ease, can be inserted through hole 34 
and then tightened to secure the camera assembly in place for operation. The 
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fastener may be loosened or removed, for example, to rotate the camera 
assembly away from the compaction roller 20 and other parts of the fiber 
placement device. 

, [0031] With further reference to Figure 1, a filter 15 can be placed 
5 on the lens 13 for filtering light in a particular manner. In one embodiment, the 
filter 15 is designed to filter light such that only the infrared component or a 
certain infrared wavelength or range of wavelengths of light can pass into the 
camera 12. Thus, the filter 15 prevents ambient visible light from entering the 
camera 12 and altering the appearance of the captured image. 
10 [0032] Other methods of filtering light can also be used to achieve 

the same, or at least similar, result. For example, the camera may be designed to 
include a built-in filter of equivalent optical characteristics. In addition, the filter 
. can be located between the camera lens 1 3 and image sensor. Alternatively, the 
camera may include an image sensor that is only sensitive in the infrared 
15 spectrum (i.e., an infrared-sensitive camera), thus eliminating the need for the 
filter. 

[0033] The light source 14 of the system 10 will now be described in 
detail. The light source 14 is positioned to emit light that illuminates the first 
portion 17 of the composite structure 22 with bright field illumination (i.e., incident 
20 light) while also illuminating the second portion 19 with dark field illumination (i.e., 
indirect light). In one embodiment, the bright field illumination "spills" over onto 
the second portion 19 and thus illuminates the second portion 19 with dark field 
illumination. 

[0034] In Figure 1, the light source 14 is shown positioned at an 
25 oblique angle 37 relative to the composite structure 22. The oblique angle 37 may 
be about forty-five degrees, although other angles are possible depending on the 
application. In addition, the light source 14 is also shown positioned to emit light 
in a direction substantially perpendicular to the direction of placement of the strips 
24 in order to highlight the defects 36, as described below. 
30 [0035] Further, the system 10 may include more than one light 

source. For example, the embodiment of Figure 2 includes two light sources 14 
positioned relative to the composite structure and compaction roller 20 on either 
side of the reflective surface 16 and camera 12. Another exemplary embodiment 
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that includes two light sources 14 is shown in Figure 4 in which two linear optical 
fiber arrays are positioned on opposed sides of the camera 12. 

[0036] In Figure 1, the light source 14 is adjustably positioned 
relative to the composite structure 22 by mounting or attaching the light source 14 
5 to a mounting apparatus 27. The mounting apparatus 27 can include a main shaft 
29, a secondary shaft 31, and a locking clamp 33 for quickly and accurately 
adjusting the position of the light source 14. The mounting apparatus 27, in turn, 
can be attached to the frame 28, to the camera 12, to the bracket 30, or to some 
other object that defines a common position for both the light source 14 and the 

10 camera 12 such that the light source 14 and camera 12 maintain a constant 
spatial relationship relative to one another. 

[0037] The quality and magnitude of the surface illumination of the 
composite structure is greatly affected by ambient lighting and by the reflectivity 
of the material. Accordingly, embodiments of the invention advantageously 

15 employ an infrared light source to more effectively illuminate dark flaws on a dark 
background. In this regard, the light source 14 can be selected from an infrared 
light or another type of light having an infrared component, such as a halogen 
light source (Figure 3) or other incandescent light sources (not shown). In other 
embodiments, the light source 14 can also include a fluorescent light source 

20 (e.g., white light LEDs, low pressure/mercury filled phosphor glass tube, etc.), a 
strobe or stroboscopic light source, a noble gas arc lamp (e.g., xenon arc, etc.), 
metal arc lamp (e.g., metal halide, etc.) and a lasers (e.g., pulsed lasers, solid 
state laser diode arrays, infrared diode laser arrays, etc.). The light from the light 
source 14 may also be pumped from through optical fibers to the point of 

25 delivery, such as is shown in Figure 4. 

[0038] In some embodiments, the light source 14 is operated at a 
power level that maximizes, or at least significantly increases, the infrared (IR) 
component of the light which works well for inspecting dark tow material, such as 
carbon. In this regard, exemplary power levels in the range of up to about one 

30 hundred fifty watts (150W) in the wavelength range of about seven hundred 
nanometers to one thousand nanometers (700nm-1000nm) have been sufficient. 
However, the particular power levels and wavelengths for the light source will 
Ijkely depend at least in part on the camera's speed and sensitivity, speed at 
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which the material is being laid, delivery losses, and reflectivity of the material 
being inspected, among other factors. For example, In other embodiments, 
wavelengths and power levels suitable for inspecting highly reflective materials 
can be employed. 

5 [0039] In the embodiment shown in Figure 1, the light source 14 

may comprise a plurality of LEDs arranged in an array or cluster formation. In one 
specific embodiment, the light source 14 includes 24 LEDs mounted in an array 
upon a three-inch square printed circuit board. 

[0040] In another embodiment shown in Figures 2 and 3, the light 

10 source 14 mc\udes four halogen light bulbs 38, although other quantities can also 
be used. To produce the dark field illumination for detecting FOD 21 , one or more 
of the bulbs 38 are either deactivated/turned off or covered with an opaque 
material. In one embodiment, the bulbs 38 are coupled to a switching device that 
allows the operator to selectively activate and deactivate individual bulbs 38 

15 and/or groups of the bulbs 38. The switching device can be implemented by 
conventional switches or by a control feature added to the user interface 76, 
described below. 

[0041] In tlie embodiment shown in Figure 4, the light source 14 
includes two linear optical fiber arrays positioned on opposite sides of the camera 

20 12. The arrays emit light supplied from a remote source (not shown) through an^^ 
optical fiber bundle 25. To produce the dark field illumination used in detecting 
FOD 21, a portion 35 of each array 14 is blocked or covered with an opaque 
material, which effectively shortens the array 14. In Figure 5, there is shown a 
linear array 14 in which the portion 35 is not covered and the entire array is fully 

25 illuminated. 

[0042] Referring back to Figure 2, the system 10 may further 
include a light reflection element 18 located near the light source 14. The 
reflection element 18 include a series of light reflecting surfaces 40 (Figure 3) that 
redirect the light towards the desired area to be illuminated. This levels the 
30 iUumir\atior) across the surf3ice and eliminates, or at least substantially reduce, 
areas of intense light (i.e., hotspots) created by the brightest portion of the light 
source 14. Hotspots are undesirable because hotspots prevent consistent 
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illumination of the composite structure, which may lead to errors during the 
processing of the images captured by the camera 12. 

[0043] The light reflection elements 40 are particularly 
advantageous for illuminating curved/contoured surfaces of composite structures 
5 because the redirection of the light permits a larger portion of the composite 
structure to be evenly illuminated. 

[0044] As shown in Figure 3, the reflection element 18 is cun/ed 
around the light source 14, such as in a parabolic shape. On the surface of the 
reflection element 18 that faces the light source 14, the reflection element 18 

10 includes curved steps 40 substantially parallel to the light source 14. The 
distance between and curvature of the steps 40 can be chosen to be sufficient to 
provide even illumination from the sum of the two light sources, one on either 
side of the region of interest. This enables the reflection element 18 to provide 
more consistent illumination of the composite structure 22, which prevents, or at 

15 least reduces, image processing errors due to inconsistent illumination of the 
composite structure 22. Alternatively, the shape and/or surface configuration of 
the reflection element 18 can be modified in other ways that also produce 
consistent illumination and scattering of the light produced by the light source 14 
over the desired portion of the composite structure 22. 

20' [0045] In an exemplary, embodiment, the reflection element 18 has 

an overall parabolic shape with seventeen parabolic curved steps 40 having a 
range of widths from about 0.125 inches at the outer edge of the reflection 
element 18 to about 0.250 inches at the center of the reflection element 18. The 
reflection element 18 also has a uniform step height of about 0.116 inches. In 

25 other embodiments, however, the reflection element may be provided with 
different numbers of steps having different uniform or varying widths and different 
uniform or varying step heights. 

[0046] Furthermore, the reflection element 18 may be adjusted in 
order to direct the light produced by the light source 14 and scattered by the 

30 reflection element 18 toward the desired portion of the composite structure. For 
example, as shown in Figure 3, the reflection element 18 is adjustably mounted 
to the mounting apparatus 27 with fasteners 42. The loosened fasteners 42 can 
move within slots 44 to correspondingly adjust the angle of the reflection element 



11 



Boeing Ref. 03^0392 
HDP Ref. 7784-000616 

18 relative to the composite structure. Once the reflection element 18 is 
positioned appropriately, the fasteners 42 are tightened to secure the reflection 
element 18 in the desired position. Adjustments of the reflection element 18 can > 
also be enabled by other methods, such as by electronic means that permit 
5 remote adjustment of the reflection element 18. 

[0047] It has been observed that the composite structure 22 
produces high glare when illuminated across the direction of placement of the 
strips 24 but produces substantially less glare when illuminated along the 
direction of placement of the strips 24. The systems and methods of at least 

10 some embodiments exploit the high-glare/low-glare phenomenon by casting light 
across the top layer of the composite strips 24 in a direction substantially 
perpendicular to the direction of placement of the strips 24. This produces a 
relatively large amount of glare on the top layer of the composite structure 22. 
The underlying layers, which produce significantly less glare than the top layer 

1 5 because of their orientation, will show through any gaps or other defects in the 
top layer and thus be easily located. In addition, twists and other surface defects 
in the top layer will alter the orientation of the strips in the top layer and thus 
correspondingly alter, i.e., decrease, the glare of the top layer at the defect 
location. 

20 [0048] While the high-glare/low-glare phenomenon occurs when 

illuminated with either visible light or infrared light, the filter 15 used in one 
embodiment of the system 10 substantially removes the glare caused by ambient 
light such that only the glare caused by the infrared light source is used to locate 
the defects and FOP. Accordingly, the filter 15 removes the interference of 

25 ambient light as the composite structure 22 is being examined for defects and 
FOD. 

[0049] In any of the system embodiments described herein, there 
may be one or more cameras 12 and/or one or more light sources 14 with or 
without reflection elements 18 (collectively referred to as light sources, 
30 hereinafter). In addition, the one or more cameras 12 and/or the one or more light 
sources 14 may be moveable relative to the composite structure. The multiple 
cameras 12 and/or multiple light sources 14 and the moveability of the camera(s) 
12 and/or the light source(s) provides system 10 flexibility in order to capture the 
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most accurate images of the composite structure. Multiple and/or moveable light 
source(s) 14 permit consistent and sufficient illumination of the desired portion of 
the composite structure, regardless of the shape of the composite structure. 
Likewise, multiple and/or moveable camera(s) 12 enable capturing an accurate 

5 image of any area of the composite structure, regardless of the shape of the 
composite structure. As such, the multiple and/or moveable light source(s) and/or 
camera(s) are particularly advantageous when illuminating and capturing images 
of and curved/contoured portions of composite structures. The multiple and/or 
moveable light source(s) and/or camera(s) are also advantageous in illuminating 

10 and capturing images of composite strips having a width that makes it difficult to 
illuminate and/or capture images of the entire strip, such that the position of the 
light source(s) and/or camera(s) may be moved over the entire strip, and/or 
multiple stationary light source(s) and/or camera(s) may be positioned to cover 
the entire strip. Systems including moveable cameras and light sources are 

15 described in detail in previously referred to U.S. Patent Application No. 
10/217,805. 

[0050] As shown in Figure 1, the system 10 can also include a 
marking device 62 for marking the location of the defects and FOD on the 
composite structure 22. The marking device 62 may be attached to the frame 28 

20 and be triggered by a processor 66 or similar device when a defect 36 or FOD 21 
that is to be reported to the operator is detected. The marking device 62 may 
spray or otherwise deposit an amount of ink, paint or the like onto the composite 
structure 22 in those areas where defects 36 and FOD has been detected. The 
markings on the composite structure 22 enables the location of the defects and/or 

25 FOD to be subsequently readily identified either automatically or manually. 

[0051] In the particular illustrated embodiment, the marking device 
62 is an Inkjet marking system that sprays a small spot of compatible ink of a 
highly visible color onto the surface of the composite structure 22 at the defect or 
FOD location to permit rapid access for repair and disposition. The marking 

30 device 62 may also be adapted to mark FOD with a different colored ink than that 
used to mark defects. Alternatively, other marking methods can also be used, 
such as a pump-fed felt-tip marker, spring-loaded marking pen, audio or visual 
alerts, and the like. 

13 
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[0052] The automated collation process includes guiding the 
composite strips 24 (Figure 1) from material creels (not shown) to an automated 
collation or fiber placement machine, such as a machine made by Cincinnati- 
Milacron and Ingersoll Milling Machines. In particular, the composite strips 24 are 
5 guided to a head unit 23 (Figure 3) and fed under a compaction roller 20. 
Focused heat energy is then applied to the incoming material and the underlying 
material that was previously laid to adhere the two materials. With the 
combination of pressure and heat, the composite strip 24 is consolidated into the 
previous layer, thus forming an additional layer of the composite structure 22. 

10 Unfortunately, defects 36 may sometimes occur during the placement of the 
composite strip 24 onto the underlying composite structure 22 and/or FOD may 
accumulate on a surface of the composite structure. 

[0053] The camera 12 and/or the reflective surface 16, which along 
with the light source 14 and any reflection element 18, can be mounted to the 

15 head unit to allow the camera 12 to continuously capture real-time images of the 
composite structure 22 and the strips 24 as the head unit moves across the 
composite structure 22 and the composite strips 24 are laid down. If the 
composite structure 22 is not planar, the inspection point should be as close to 
the nip point as possible, as described above. If the composite structure 22 is 

20 planar, the inspection point can be located further from the placement head unit. 
In either case, the images can be stored in a memory device 64 for future 
analysis and/or processed immediately by the processor 66, as discussed more 
fully below. 

[0054] Figure 6 shows an exemplary raw or unprocessed camera 
25 image 68, which includes a plurality of pixels having a range from black through a 
plurality of shades of gray to white. The image 68 includes a bright field 69 and a 
dark field 71. The bright field 69 corresponds to the first portion 17 (Figure 1) of 
the composite structure illuminated with the bright field illumination, whereas the 
dark field 71 corresponds to the second portion 19 (Figure 1) illuminated with the 
30 dark field illumination. 

[0055] The bright field 69 in the unprocessed camera image 68 
illustrates a contrast between a potential defect 75, such as a tow gap, and the 
remaining portions of the composite structure 22 that are defect free. In the 
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illustrated embodiment of the bright field 69, the potential defects are shown as 
black or gray areas 70, while the remaining non-defective portions of the 
composite structure 22 remain substantially white 72. Once the potential defects 
are located, however, the potential defects may still require further processing to 
5 determine if the potential defects are acceptable or unacceptable, as described 
below. 

[0056] The dark field 71 of the unprocessed camera Image 68 
illustrates a contrast between the dark field illumination that is reflected by FOD 
and the remaining portions of the composite structure 22 that are FOD free. In 

10 the illustrated embodiment of the dark field 71, a resin ball is indicated by way of 
two substantially white reflection spots 77 visible on either side of the resin ball. A 
potential fuzz ball is shown as a substantially white area 79 indicative of the dark 
field illumination being reflected from the edges and fiber ends of the fuzz ball. 
The remaining FOD-free portions of the composite structure 22 remain black or 

15 gray. 

[0057] Although resin balls and fuzz balls are visually different in the 
image 68, further processing may still be required to differentiate the resin balls 
from the fuzz balls, for example, to allow the marking device 62 (Figure 1) to 
mark resin balls with a different colored ink than that used to mark fuzz balls. To 

20 differentiate resin balls from fuzz balls, a known "blob" imaging process can be 
used in which a series of decisions are made based upon mathematical 
operations and comparisons to established criteria for specific geometries. One 
differentiator that can be used to distinguish between resin balls and fuzz balls is 
edge roughness because fuzz balls have greater edge roughness than resin 

25 balls. 

[0058] Further processing may also be needed to determine 
whether a potential FOD anomaly is acceptable or unacceptable according to 
maximum allowable dimensional parameters. That is, the system may only mark 
or flag FOD which exceed the maximum allowable dimensional parameters. To 
30 make this determination, the system may count the number of pixels within a 
particular image region that represents the reflected dark field illumination by a 
potential FOD anomaly. The system may then use the pixel count to compute an 
indirect quantitative measurement for the particular FOD anomaly based upon 

15 



Boeing Ref. 03-^0392 
HDP Ref. 7784-000616 

correlation data including a predetermined relationsiiip between pixel count and 
FOD size. 

[0059] With further reference to Figure 1, the processor 66 may 
receive the images 68 from the camera 12 or from the memory device 64 in 
5 which the images 68 have been stored. The processor 66 may then process and 
analyze the images to facilitate the reliable' detection of defects and FOD. In at 
least one embodiment, the processor 66 and memory device 64 are components 
of a conventional computer. 

[0060] The system 10 may also includes a user interface 76 that is 
10 in communication with the processor 66. The user interface 76 can be 
programmed such that it can run from a wide range of software applications, 
including but not limited to DOS, Windows 98, Windows/NT, Windows 2000, 
Windows CE, Linux, Unix, and equivalents. 

[0061] As shown in Figure 7, the user interface 76 includes a 
15 display screen 80, such as on a computer monitor, and can also include an Input 
device, such as a keyboard and mouse (not shown), for permitting an operator to 
move a cursor about the display screen 80 and input various system settings and 
parameters. The display screen 80 can also be touch-sensitive for permitting the 
operator to input the desired settings by manually touching regions of the display 
20 screen. 

[0062] The user interface 76 includes a window 81 in which an 
image 74 of the composite structure 22 is displayed for viewing by the operator or 
other user. Although the image 74 can be the unprocessed camera image 68 
(Figure 6), the image 74 shown in Figure 7 is a processed image in which the 

25 rectangular region 73 has been binarized. During binarization, all shades of gray 
above a predetermined threshold value can be changed to white, while all gray 
shades below the threshold are changed to black to heighten the contrast of 
defects and improve the accuracy of defect detection. In other embodiments, the 
binarization step need not be performed but instead the raw image, rates of 

30 change of the light levels in the raw image, and/or color changes in the images 
can be used to identify the defects and FOD. 

[0063] The user interface 76 also provides user controls 78 for 
allowing various user inputs to the system. In the particular illustrated 

16 
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embodiment of Figure 7, the user interface 76 allows adjustment to the 
binarization threshold. Generally, the setting of the binarization threshold involves 
a tradeoff between the sensitivity with which defects are detected and the 
resolution with which the defects are depicted. In one embodiment, the 
5 binarization threshold is set to about 128 which corresponds to the mid-point on 
the 8-bit digitizing range of 0 to 255. However, other binarization threshold 
values can be employed depending at least in part on the particular application, 
available lighting, camera settings, among other factors. 

[0064] The user controls 78 also allow the user to adjust or shift the 

10 viewing area within the window 81. During operation, the window 81 displays 
real-time moving video images of the illuminated portions 17 and 19 of the 
composite structure 22 as the camera 12 and/or the reflective surface 18 are 
moved relative to the composite structure 22. By accessing the user controls 78, 
the user can shift or adjust the window 81 such that the window 81 

15 simultaneously displays both the bright field 69 and the dark field 71 . 

[0065] The interface 76 may also allow the user to input the 
maximum allowable dimensional parameters for acceptable defects and FOD 
and/or input the acceptable tolerances of the maximum dimensional parameters, 
among others. 

20 [0066] In addition to displaying images of the composite structure 

22, the display screen 80 also includes a defect table 82 which lists the 
discovered defects and provides information for each defect, such as location, 
size, and the like. The display screen 80 can also include an FOD table (hot 
shown) for listing and providing Information about the discovered FOD anomalies. 

25 The display screen 80 can further include status indicators 84 that notify the user 
whether a particular image area is acceptable or not acceptable based on 
predefined criteria, such as maximum allowable dimensional parameters and 
tolerances. 

[0067] Figure 8 illustrates another embodiment of the user interface 
30 76 that may be used in a system 10 that includes two or more cameras 12 for 
capturing images of the composite structure 22. As shown, the user interface 76 
includes a display screen 86 that displays images 88 and 90 of the composite 
structure 22 for viewing by the operator or other user. The image 88 is captured 

17 
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by the first camera and represents the portion 17 of the composite structure 22 
illuminated with bright field illumination. The dark field image 90 is captured by 
the second camera and represents the portion 19 of the composite structure 22 
illuminated with dark field illumination. Alternatively, multiple user interfaces can 
5 be used to display the images of multiple cameras 12 in which each interface 
displays an image from one or more cameras and presents one or more links to 
the user interface(s) that display images from the other camera(s). Images from 
multiple cameras mounted side by side can be assembled to form a composite 
view of a larger area of the composite structure. 

10 [0068] In another form, the present invention provides methods for 

identifying FOD and defects during fabrication of a composite structure. In one 
embodiment, the method generally comprises: illuminating a portion of the 
composite structure with bright field illumination; illuminating another portion of 
the composite structure with dark field illumination; acquiring an image of the 

15 illuminated portions of the composite structure; analyzing the image to identify 
defects in the portion of the composite structure Illuminated by bright field 
illumination; and analyzing the image to identify foreign objects and debris on the 
another portion of the composite structure illuminated by dark field illumination. 

[0069] Accordingly, embodiments of the present invention can/ 

20 operate more efficiently with fewer interruptions than conventional fiber 
placement systems because human intervention is not required for inspection of 
the composite structure for defects and foreign objects and debris. Instead, the 
embodiments of the present invention can rapidly detect and identify defects and 
FOD so that the same can be subsequently readily identified during remedial 

25 actions to repair the defects and/or remove the FOD, which would otherwise 
create structural flaws or inconsistencies that may affect the integrity of the 
composite structure. As such, less material is wasted, less labor is expended in 
inspection, and less machine downtime is incurred during the fabrication process; 
therefore, a lower cost composite structure is achieved on average. Additionally, 

30 embodiments also enable an improvement in the overall quality of the parts 
produced because defects and FOD can be detected more uniformly and reliably 
with the various automated systems and methods of the^ invention than a 
traditional human inspection. 
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[0070] The description of the invention is merely exemplary in 
nature and is in no way intended to limit the invention, its application, or uses. 
Thus, variations that do not depart from the substance of the invention are 
intended to be within the scope of the invention. Such variations are not to be 
regarded as a departure from the spirit and scope of the invention. 
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